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Why GAO Did This Study 

The fiscal year 1993 National 
Defense Authorization Act 
established the National Guard 
Youth Challenge Program as a pilot 
program to evaluate the 
effectiveness of providing military 
based training to improve the life 
skills of high school dropouts. The 
Assistant Secretary of Defense for 
Reserve Affairs, under the 
authority of the Under Secretary of 
Defense for Personnel and 
Readiness, is responsible for 
overall policy for the program. The 
National Guard Bureau (NGB) 
provides direct management and 
oversight. In 1998, Congress 
permanently authorized the 
program and began decreasing the 
federal cost share until it reached 
its current level of 60 percent in 
2001. Conference Report 108-767 
directed GAO to review the 
program. Specifically, GAO 
reviewed (1) historical trends of 
the program; (2) the extent of 
analyses performed to determine 
program costs and the need to 
adjust the federal and state cost 
share; and (3) NGB oversight of the 
program. GAO is also providing 
information on Reserve Affairs’ and 
states’ efforts to obtain funding 
from alternative sources. 



What GAO Recommends 



GAO recommends a number of 
actions designed to improve the 
management and oversight of the 
National Guard Youth Challenge 
Program. DOD concurred with 
GAO’s recommendations. 



www.gao.gov/cgi-bin/getrpt7GAO-06-140. 

To view the full product, including the scope 
and methodology, click on the link above. 

For more information, contact Sharon Pickup 
at (202) 512-9619 or pickups@gao.gov. 



What GAO Found 

Between fiscal years 1998 and 2004, total expenditures for the Challenge 
Program, including funds spent to cover the federal and state cost shares 
and federal management expenses, have increased from about $63 
million to $107 million. During this same period, participation in the 
program has grown from 10 sites in 10 states to 29 sites in 24 states and 
Puerto Rico. Since the program’s inception, NGB has reported positive 
performance outcomes in academic performance, community service 
activities, and post-residential placements. For example, in 2004, NGB 
reported graduating 7,003 students, or 79 percent of those enrolled, with 
70 percent of those graduates earning a high school equivalent diploma. 

While Reserve Affairs and NGB have expressed concern about the 
current program funding level and have suggested increasing both the 
cost basis used to determine funding needs and the federal cost share, 
neither has performed analyses to support the need for such changes. 
Federal financial standards state that reliable cost information is crucial 
for effective management of government operations. Since 1993, NGB 
has used $14,000 per student as the basis for determining the amount of 
funds needed to cover program operating costs, and applied the federal- 
state cost share to this amount. To keep pace with inflation, NGB has 
suggested increasing the per student cost to $18,000. Reserve Affairs has 
reported some states are having difficulty meeting their share and, in 
2004, recommended the federal share be increased from 60 percent to 75 
percent. However, neither Reserve Affairs nor NGB has compiled or 
analyzed data on actual program costs, states’ financial situations, or the 
impact of adjusting the federal and state cost-share. Without better cost 
and financial information, the Department of Defense (DOD) cannot 
justify future funding requests or a change in the cost-share ratio. 

Although NGB uses various oversight mechanisms, it lacks a complete 
oversight framework, making it difficult to measure program 
effectiveness and to adequately address audit and review findings. Also, 
some audits have not been performed as required. The Government 
Performance and Results Act suggests a complete oversight framework 
including goals and measures against which to objectively evaluate 
performance. While NGB requires states to report certain performance 
outcomes, it does not require states to establish performance goals in 
these areas, and therefore does not have a firm basis for evaluating 
program outcomes and DOD’s return on investment. Existing 
agreements require state programs to be audited at least every three 
years. However these audits have not been conducted as required and no 
provisions exist for submitting audit results to NGB. Without regular 
audits and access to results, NGB cannot be assured that programs are 
using federal funds appropriately and that audit findings are addressed. 
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Accountability * Integrity * Reliability 



United States Government Accountability Office 
Washington, DC 20548 



November 29, 2005 

The Honorable John Warner 
Chairman 

The Honorable Carl Levin 
Ranking Minority Member 
Committee on Armed Services 
United States Senate 

The Honorable Duncan L. Hunter 
Chairman 

The Honorable Ike Skelton 
Ranking Minority Member 
Committee on Armed Services 
House of Representatives 

After Operation Desert Storm and the end of the Cold War, Congress 
began to reevaluate the focus of U.S. military activities and proposed using 
some military assets and training to help address critical domestic 
challenges such as drugs, poverty, and unemployment. In particular, some 
policymakers saw an opportunity to use the military’s capabilities to 
address domestic needs by providing role models for youth and training 
and educational opportunities for the disadvantaged. Section 1091 of the 
National Defense Authorization Act for Fiscal Year 1993 established the 
National Guard Youth Challenge Program (Challenge Program) as a pilot 
program to evaluate the feasibility and effectiveness of military based 
training to improve the life skills and employability of high school 
dropouts. After the program was permanently authorized in 1998, 

Congress began limiting the share of state operating costs covered by the 
federal government through the Department of Defense (DOD), which 
initially paid for 100 percent of these costs. DOD now pays 60 percent of 
state operating costs, and states provide 40 percent. In addition, the 
authorizing legislation for the Challenge Program allows the Secretary of 
Defense to use nondefense funding sources in support of the program. 

The Assistant Secretary of Defense for Reserve Affairs, under the authority 
of the Under Secretary of Defense for Personnel and Readiness, is 
responsible for management oversight of the Challenge Program, including 
policy and funding issues, while the National Guard Bureau (NGB) is 
responsible for the administration of the program, including daily 
management and oversight. For example, NGB maintains frequent 
contacts with state programs and conducts program evaluations through a 
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contractor, while Reserve Affairs prepares the program budget based on 
input from NGB and ensures that states are able to provide their required 
share. To participate in the program, states must sign a cooperative 
agreement with NGB that lays out roles and responsibilities among the 
states and NGB, and also provides guidance on the content of the 
program. Since 1993, NGB has used a cost of $14,000 per student as a 
basis for providing funds to state programs. 

Conference Report 108-767 directed the Comptroller General to conduct a 
study of the Challenge Program. 1 Specifically, we examined: (1) historical 
trends of the Challenge Program, including program expenditures, 
participation, and performance; (2) the extent to which Reserve Affairs 
and NGB have determined actual program costs and the consequent need 
to adjust the federal and state cost share; and (3) the extent to which NGB 
has provided oversight of the program. We also determined the extent to 
which Reserve Affairs and participating states have made an effort to 
obtain alternative funding support for the Challenge Program. 

In performing our work, we interviewed officials from the Office of the 
Assistant Secretary of Defense for Reserve Affairs; NGB; the Departments 
of Education, Labor, and Justice; the contractor that monitors and 
evaluates state programs; the National Guard Youth Foundation; and 
selected states. We also reviewed the program’s annual reports and other 
documents provided by Reserve Affairs and NGB as well as the data 
management system used by states to collect Challenge Program student 
information and track individual and program activities. We concluded 
that the data were sufficiently reliable for our purposes. We analyzed 
program evaluations, resource management reviews, and audits, in 
addition to reviewing existing policies and procedures for managing the 
program. We conducted a survey of all 29 established programs to collect 
information on their budgets and expenditures, federal and state funding 
levels, each state’s ability to fund their individual program, and the 
availability of funding from other sources. We also visited or contacted 
eight Challenge Program sites in seven states and asked them about any 
alternative sources of funding for their programs. A detailed description 
of our scope and methodology is presented in appendix I. We conducted 
our work from January 2006 to October 2006 in accordance with generally 
accepted government auditing standards. 



Conference Report 108-767 accompanied the Ronald W. Reagan National Defense 
Authorization Act for Fiscal Year 2005, Pub. L. No. 108-375 (2004). 
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Results In Brief 



According to NGB, program expenditures and state participation in the 
Challenge Program have increased since the program began, and the 
program has achieved positive performance outcomes. Between fiscal 
years 1998 and 2004, total expenditures for the Challenge Program, 
including funds spent to cover the federal and state cost shares and federal 
management expenses, have increased from about $63 million to about 
$107 million. For fiscal year 2004, for example, NGB spent approximately 
$61.6 million for the federal share of the program and $6.8 million for NGB 
management costs, while states contributed approximately $40.5 million. 
Ten states participated in the first year of the pilot program in 1993; and 
today there are 29 Challenge Program sites in 24 states and Puerto Rico. 
Additionally, 9 states remain on a waiting list to start new programs, and 
several states have expressed an interest in expanding their existing 
programs. Some common performance outcomes that NGB reports 
include the number of graduates earning a general educational 
development credential, changes in program students’ scores on 
standardized math and reading tests, and the number of community 
service hours performed by Challenge Program students. For example, 
the Challenge Program reported that 70 percent of graduates in 2004 
earned a general education development credential. Another performance 
outcome that NGB reports is the number of placements of program 
graduates in continuing education, the military, or the labor force at the 
time they complete their 12-month post-residential phase. For example, in 
2004, 3,698 graduates were placed in one of these categories by the end of 
the post-residential phase of the program. 

Although Reserve Affairs and NGB have expressed concern about the 
current program funding level and have suggested increasing both the cost 
basis used to determine funding needs and the federal cost share, we 
found that neither Reserve Affairs nor NGB has performed analyses to 
support the need for such changes. Good budget practices, included in the 
Office of Management and Budget’s Federal Financial Accounting 
Standards, state that agencies should determine actual costs of their 
activities on a regular basis and that reliable cost information is crucial for 
effective management of government operations. Since 1993, NGB has 
used a cost of $14,000 per student as the basis for determining the amount 
of funds needed to cover program operating costs. However, due to cost 
variations between states, program officials that we surveyed reported 
that they actually spent between $9,300 and $31,031 per graduate. 
According to NGB, all 29 programs are providing the services required by 
the cooperative agreements, and several states have added program 
enhancements such as field trips or vocational classes. However, some of 
these states reported that they reduced nonrequired services to stay within 
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their budgets. For example, they implemented staff pay and hiring freezes, 
eliminated the student stipend, and eliminated some enrichment activities 
such as field trips and vocational classes. In addition, Reserve Affairs, 
NGB, and participating states have suggested that the cost-share ratio be 
changed from its current 60 percent federal share to a 76 percent federal 
share because they believe that the current required 40 percent state share 
is sometimes difficult for states to meet. However, neither Reserve Affairs 
nor NGB has compiled or analyzed data on actual program costs, states’ 
financial situations, or the impact of adjusting the federal and state cost 
share. Reserve Affairs has asked the National Guard Bureau to determine 
a new funding formula for the program based on individual state needs, 
but, at the time of our review, NGB had not yet done so and Reserve 
Affairs has not given NGB a deadline for completion. Without better cost 
and financial information, DOD cannot justify future funding requests or a 
change in the cost share ratio. 

Although NGB has several mechanisms in place for overseeing the 
Challenge Program, it lacks a complete oversight framework, making it 
difficult to measure the effectiveness of the program and to adequately 
address audit and review findings. NGB conducts several oversight 
activities, including informally communicating with state program 
directors and having an outside evaluator conduct yearly operational 
evaluations and biennial resource management reviews of the Challenge 
Program, and these reports have prompted some changes to the program. 
However, until recently, NGB did not have a formal mechanism for 
tracking the findings of these reports. In response to our review, NGB 
recently implemented a mechanism for tracking these findings. As part of 
its oversight activities, NGB does require each state program to report 
certain performance outcome measures, such as academic test scores. 
However, it does not require states to establish performance goals in these 
areas. A complete oversight framework includes performance goals and 
measures against which to objectively measure performance as well as a 
mechanism for tracking findings of audits or reviews and responding to 
those findings. Without clear and agreed upon performance goals, there is 
no objective yardstick against which to fully measure program 
performance and effectiveness and thereby assess DOD’s return on 
investment. In addition, Standards for Internal Control in the Federal 
Government states that agencies need to ensure that the findings of audits 
and reviews are promptly resolved. Under the cooperative agreements 
governing the Challenge Program, United States property and fiscal 
officers in each state are required to conduct full audits of the individual 
Challenge Programs at least once every three years. However, according 
to property and fiscal officers that we spoke with, all of these audits have 
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not been conducted as required, due to the low priority placed on these 
audits and lack of staff for the property and fiscal officer. If these audits 
are not conducted, it may be difficult to ensure that federal interests are 
adequately protected. Copies of these audits are not currently being 
provided to program managers at NGB because, according to the Chief of 
Property and Fiscal Affairs at NGB, there is no specific requirement to do 
so. If the National Guard does not see the audits, it cannot ensure that the 
findings of these audits are promptly resolved. 

The authorizing legislation for the Challenge Program allows the Secretary 
of Defense to use nondefense funding sources in support of the program, 
and allows states to supplement program funds from other resources. 
However, Reserve Affairs has not adopted a formal strategy for pursuing 
nondefense funding, while some states have been successful in securing 
alternative funding support for their programs. Reserve Affairs has 
primarily adopted informal strategies to contact agencies outside DOD to 
inform them about the Challenge Program and seek opportunities for 
partnerships, but has not been successful in securing funds from these 
agencies. We found that because Reserve Affairs has not made a formal 
business case to request funds from these nondefense agencies, these 
agencies are unable to determine whether or not they are specifically able 
to fund the Challenge Program. Officials from some federal agencies told 
us that that their agencies have general authority to provide funds to other 
programs if those programs are consistent with their agency’s interests. 
However, officials at the Department of Justice, the Department of Labor, 
and the Department of Education stated that Reserve Affairs needed to 
present more specific information to them before they could determine 
whether funds could be provided to the Challenge Program. According to 
those officials, at a minimum, any such request should contain the amount 
of funding sought and a sufficiently detailed description of the program to 
allow the agency receiving the request to determine whether it would be 
an appropriate use of funds. Until Reserve Affairs makes a more formal 
request for funding, other agencies will be unable to determine if they can 
provide funds for the Challenge Program. At the state level, some states 
we contacted had made efforts to obtain alternative funding support to 
enhance or maintain their program’s operations. For example, some state 
programs organized into charter schools to obtain funding from local 
school districts, applied for National School Lunch Program funding, 
obtained grants from nonprofit organizations, and received additional 
funding from the state government beyond the required match. Some 
states reported obstacles in securing supplemental funding, such as the 
prohibition against using Challenge funds to hire grant writers. 
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We are making recommendations designed to improve DOD’s management 
and oversight of the Challenge Program and to strengthen efforts in 
obtaining alternative funding support for the program. In commenting on 
a draft of this report, DOD concurred with our recommendations. DOD’s 
comments and our evaluation of them appear later in this report. 



Background 



The National Guard Youth Challenge Program is a 17-month program that 
serves at-risk youth at 29 sites in 24 states and Puerto Rico. The purpose 
of the program is to improve the education, life skills, and employment 
potential of students by providing military-based training, supervised work 
experience, and knowledge in eight core program components. 2 Students 
must be 16 to 18 years old, drug-free, unemployed, high school dropouts, 
and not in trouble with the law. NGB reports that more than 59,000 
students have been graduated from the Challenge Program since it began 
as a pilot in 1993. The program was authorized by 32 U.S.C. §509 on a 
permanent basis in fiscal year 1998, at which time states were to begin 
paying a share of operating costs. 



Challenge Program Each Challenge Program site operates two residential classes per year, 

Structure one °f Which begins in January and the other around July. 3 A typical 

graduation goal is 100 students per class, or 200 per year, although several 
programs graduate more students. In 2004, for example, Illinois graduated 
almost 800 students in 2004, and Louisiana’s three sites combined 
graduated more than 950 students. The residential phase of the program 
runs 22 weeks and includes a 2-week Pre-Challenge phase. During Pre- 
Challenge, applicants are assessed for their ability and motivation to 
complete the remaining 20 weeks of the residential program. Those who 
successfully complete Pre-Challenge are then formally enrolled in the 
Challenge Program in numbers that equal each program’s graduation 
target plus normal program attrition rate. In the residential phase, 
students receive military-based training and supervised work experience. 
Additionally, each state develops a curriculum that incorporates the eight 
core components and the tasks, conditions, and standards that students 



" The eight core program components are: leadership/followership, job skills, responsible 
citizenship, health and hygiene, service to the community, academic excellence, life-coping 
skills, and physical fitness. 

3 NGB reports on site activities by NGB class numbers. The first class under the pilot 
program, NGB-1, began in July 1993, and NGB-25 started in July 2005. 
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must complete to demonstrate progress in those components. 4 Each 
student must receive a score of at least 80 percent on each core 
component to be graduated from the program. 

During the 12-month post-residential phase, individuals who have 
successfully completed the residential phase are involved in placement 
activities, which include employment, education, volunteer activities, or 
any combination of the three or military service. The graduates work with 
adult mentors who were matched with them during the residential phase. 
These mentors provide guidance and support to the graduates and are 
required to contact the youths twice each month at a minimum. Program 
staff use the written, post-residential action plan that each student 
prepares and updates during the residential phase to monitor placement 
activities. Mentors also use this plan during their interactions with 
graduates. The Challenge Program reports youth placement activities at 
the end of the 12-month follow-up period. To further assess the long-term 
impact of the program, NGB has contracted with AOC Solutions to 
conduct a retrospective longitudinal study of program graduates as well as 
students who did not complete the program. 



Program Funding Prior to 1998, the federal government, through DOD, completely funded 

the Challenge Program. In fiscal year 1998, Congress began requiring 
states to provide a minimum of 25 percent of their programs’ operating 
costs. The state cost share increased 5 percent each year until fiscal year 
2001, when it reached the current funding requirement of 40 percent. 5 
Although some states had provided more funds than required in the past, 
program funding each year is now determined by the 40 percent share, 
which is based on $14,000 for each youth targeted for graduation. In 
addition to the federal and state funds used to operate the program sites, 
DOD also provides funds for NGB management expenses such as program 
evaluations, contractor-provided training, and travel for training and 
workshops. These NGB program management costs are not subject to the 
federal/slate 60/40 cost share requirement. 



4 A task is a clearly defined and measurable activity accomplished by an individual. 
Condition describes the training situation, environment, or field conditions under which 
the cadet must perform the task. The standard establishes criteria for how well a task must 
be performed. 

5 32 U.S.C. §509(d)(4). 
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Each state submits a budget to NGB that is based on that state's target for 
number of graduates. Since the program’s inception, the funding provided 
by NGB has been based on a cost per student of $14,000. 6 For example, if 
a state has a target of 100 students per class (200 per year) to graduate, the 
estimated program costs would be $2.8 million. The federal contribution, 
or 60 percent of the total, would be $1.68 million, while the state 
contribution would be $1.12 million. 

To receive federal funding, a state must certify that it has sufficient funds 
to provide its 40 percent share. State funds can be composed of cash, 
noncash supplies, services, or a combination of these sources. States are 
allowed to provide additional funding (over and above the 40 percent 
share) to the program from sources such as individual and corporate 
donations, additional moneys from the state general fund or other state 
revenue sources, or other federal funding. Some Challenge Program sites, 
for example, operate as alternative schools and are reimbursed by their 
state education agencies for portions of their program costs. 



Management, Oversight, Reserve Affairs, under the authority of the Under Secretary of Defense for 
and Support Personnel and Readiness, is responsible for preparing the annual budget 

and reviewing state budgets and funding certifications. Reserve Affairs is 
to monitor program compliance with DOD policy, issue supplemental 
policy guidance, and submit the Challenge Program annual report to 
Congress. NGB provides day-to-day administration and oversight of the 
Challenge Program, issuing regulations, and submitting budgets and 
annual report drafts to Reserve Affairs. NGB has contracted with AOC 
Solutions to assist with the oversight of the Challenge Program. AOC 
Solutions performs the annual operational evaluations and the biennial 
resource management reviews. This contractor also pulls together the 
program information for the annual report and maintains and oversees the 
Data Management and Reporting System (DMARS), which is used to 
collect student data and report on individual and program activities. NGB 
has also contracted with Dare Mighty Things to provide training and 



6 Funding allocations to the states have historically been based on target enrollments; that 
is, the maximum number of students per class that are funded annually in a program’s 
budget. Beginning with fiscal year 2004, funding allocations are now based on target 
graduation, which is the maximum number of students identified in a program’s budget for 
participation in and graduation from the program. According to NGB officials, although 
allocations were originally based on a site’s target enrollment, this figure was treated as a 
graduation goal. These numbers differ by state. 
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technical assistance to Challenge Program staff. 7 Finally, NGB has United 
States property and fiscal officers in each state who are responsible for 
receiving and accounting for all federal Challenge Program funds and 
property under control of the National Guard in that state. The property 
and fiscal officers are also responsible for ensuring that federal funds are 
properly obligated and expended. 

NGB enters into cooperative agreements with governors of states 
approved to participate in the Challenge Program. The cooperative 
agreements describe the responsibilities of the states and NGB as well as 
the funding, costs, and regulations for operating National Guard Youth 
Programs. 8 The cooperative agreements also define the eight core 
components and provide guidance on how to run the residential and post- 
residential phases and other aspects of the Challenge Program. Each 
Challenge Program state is also required to submit state plans and budget 
estimates for their state. These state plans must include details on the 
state’s procedures and be consistent with overall program guidance 
provided by DOD. For example, state plans include information on 
application and selection procedures, staffing and staff training, and a 
detailed budget. 



Challenge Program 
Expenditures and 
Participation Have 
Increased and NGB 
Has Reported Positive 
Performance 
Outcomes 



According to NGB, Challenge Program expenditures and state 
participation have increased since the program began, and the program 
has achieved positive program performance outcomes over time. Since 
the program’s inception, total expenditures have increased from about $63 
million to about $107 million per year. The number of states participating 
in the Challenge Program has also increased, and several states have 
expressed interest in adding a program or expanding existing ones. 
Challenge sites must account for their activities throughout the year, and 
NGB has reported positive performance outcomes over time. 



Contracts with AOC Solutions and Dare Mighty Things were competitively awarded for a 
1-year base period with four 1-year options. 

8 The Master Youth Programs Cooperative Agreement covers two youth programs: the 
National Guard Youth Challenge Program and the National Guard STARBASE Program. 
The STARBASE Program was established to improve the knowledge and skills of at-risk 
youth in math, science and technology. 
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Program Expenditures NGB reports that overall federal expenditures for the Challenge Program 

Have Increased Over Time have increased over time, but states have also increased their 

expenditures since the program was permanently authorized in fiscal year 
1998. Between fiscal years 1998 and 2004, total expenditures for the 
Challenge Program, including funds spent to cover the federal and state 
cost shares and federal management expenses, have increased from about 
$63 million to about $107 million. For fiscal year 2004, for example, NGB 
expenses included $61.6 million for the federal cost share and $5.8 million 
for NGB management costs, while states contributed approximately $40.5 
million. 9 In addition, in 2000 and 2001, the Challenge Program received $5 
million and $7,483,500 respectively from the Department of Justice. 
Reserve Affairs stated that the primary use of these funds has been to start 
new Challenge Program sites. Since 2001, four programs were established 
using these funds and three programs’ operations were maintained in 2002. 
In total, approximately $5.97 million remain unspent in a nonexpiring 
account. Officials at Reserve Affairs and NGB told us that these funds 
remain unspent because no new Challenge Programs have started. 
According to these officials, new programs have not been established 
because state governments have not committed the required 40 percent 
match. (See fig. 1 for total program expenditures from fiscal year 1998 to 
2004, broken down by federal and state cost share and NGB management 
expenses.) 



9 At the time of our review, final expenditure reports for fiscal year 2005 were unavailable. 
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Figure 1 : Annual Expenditures for the National Guard Youth Challenge Program 
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Source: GAO analysis of Reserve Affairs and NGB data. 



Note: Federal cost shares for fiscal years 2001 and 2002 include $6.5 million from Department of 
Justice. State cost share amounts are the required state matches only and do not include any 
additional funding or in-kind support the states might have provided. 



The Number of States 
Participating in the 
Challenge Program Has 
Increased 



When the pilot program began in 1993, there were 10 Challenge Program 
sites in 10 states. The program has now grown to 29 sites in 24 states and 
Puerto Rico. 10 In fiscal year 2005, Wyoming received funds to start up a 
program site. According to NGB, Wyoming will begin its first class in 
January 2006. In addition to those states currently operating Challenge 
Program sites, there are also nine states that have expressed interest in 



10 Louisiana has three Challenge Program sites, and Georgia and South Carolina each 
operate two sites. 
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establishing new programs. 11 For example, according to NGB officials, 
representatives from Washington and Indiana National Guard units have 
visited some existing program sites and are in the process of developing 
state programs. Other states are interested in expanding their programs to 
serve more youth at existing sites or to open new locations. On the other 
hand, for various reasons including difficulty meeting the state match 
requirement, lack of state support, and substandard facilities, four states 
have discontinued their Challenge Programs. Connecticut, a pilot program 
state, dropped its program in 1994 after two classes. Colorado 
discontinued its program after 1999, Missouri after 2002, and New York 
after 2003. Figure 2 describes the number of Challenge Program sites for 
each year since the program began. Appendix II identifies the individual 
states with Challenge Program sites. 



Figure 2: Number of National Guard Youth Challenge Program Sites by Reporting Year 

Number of sites 
35 




Reporting year 

Source: GAO analysis of data from AOC Solutions, Inc. 



Note: Reporting year is the time frame covered in annual reports. It includes the July class from the 
previous year and the January class from the current year. 



n According to NGB, as of August 2005, the District of Columbia and eight states had 
expressed interest in starting a Challenge Program: Alabama, Colorado, Connecticut, 
Delaware, Indiana, Nevada, Pennsylvania, and Washington. 
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Student participation in and graduation from the Challenge Program have 
also increased over time. States are required to track the number of youth 
who have applied to the Challenge Program, enrolled in the third week of 
the program (after the 2-week Pre-Challenge phase), and were graduated 
from the residential phase. According to NGB, the target graduation rate 
for 2004 was 6,961; the actual number of enrollees was 8,920; and 7,003 
students were graduated, or 79 percent of those enrolled, from the 
Challenge Program. Figure 3 shows the target numbers, the actual number 
of students who were enrolled in the residential phase at week 3, and the 
number that graduated from the program from 2000 through 2004. 



Figure 3: Youth Participation in National Guard Youth Challenge Program 

Number of participants 
10000 




Reporting year 



Target enrollment 
Actual enrollment 
Number of graduates 



Source: GAO analysis of data from AOC Solutions, Inc. 



Note: Reporting year is the time frame covered in annual reports. It includes the July class from the 
previous year and the January class from the current year. 

Although all Challenge Programs graduate two classes per year, the 
number of graduates per class varies. In addition, some states have 
multiple programs. For example, Louisiana has three Challenge Programs 
and, in 2004, graduated a total of 952 students. Figure 4 identifies those 
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states currently participating in the Challenge Program by the number of 
graduates they reported for 2004. 



Figure 4: Challenge Program Sites by 2004 Graduation Results 




cp 




No program 

200 or fewer graduates 

201 to 400 graduates 
More than 400 graduates 
Challenge Program Sites 



Source: GAO analysis of data from AOC Solutions, Inc. 

Note: The 2004 reporting year covers NGB classes 21 and 22, which began in July 2003 and January 
2004. 
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NGB Has Reported 
Positive Performance 
Outcomes 



NGB has reported positive performance outcomes in academic 
performance, community service activities, and post-residential 
placements. Program performance information is tracked by each 
Challenge site and submitted to NGB. Each year, the Challenge Program 
reports on outcomes for the two classes completing the 22-week 
residential phase during that reporting year and for the two preceding 
classes as they complete their 1-year post-residential follow-up phase. The 
Challenge Program sites use the same automated system, DMARS, to 
collect information on students and report on their progress and activities. 
The information collected in DMARS is reviewed by the contractor 
through weekly and monthly reports and during random checks of source 
documents during operational evaluation site visits. 

Some residential phase outcomes of the Challenge Program, such as the 
number of graduates earning a general educational development (GED) 
credential or high school degree and changes in scores on standardized 
math and reading tests, are tied to the core component of Academic 
Excellence. 12 For example, NGB reported that 70 percent of graduates in 
2004 earned a GED. Figure 5 illustrates the outcomes of GED attainment 
for the past 5 reporting years. Students also take the Tests of Adult Basic 
Education, a series of tests that identify individual education levels in 
various academic subject areas. Each state program tests its students early 
in the residential phase and then toward the end of the 22-week period, 
and it reports the changes in test scores. In 2004, for example, NGB 
reported that graduating students improved 1.7 grade levels in reading and 
1.8 grade levels in math during the residential phase. Another core 
component, Service to the Community, requires each student to perform a 
minimum of 40 hours of service to the community or conservation project 
activities. The number of community service hours performed by each 
student is tracked, and the total number of hours for each site is another 
outcome that the Challenge Program reports annually. For example, in 
2004, NGB reported that Challenge Program students performed more 
than 590,000 hours of community service, such as maintaining historical 
cemeteries and parks and supporting organizations such as Special 
Olympics and Habitat for Humanity. 



12 In some states, younger youths are not eligible to take the GED due to age restrictions, 
and some youths are not academically prepared to take the tests. Some programs offer 
alternative degrees, such as adult education diplomas; some programs are accredited high 
schools and can offer high school diplomas or credits; and many youths return to their 
home high schools to earn their diplomas. 



Page 15 



GAO-06-140 Defense Management 




Figure 5: Number of GEDs Earned by Challenge Students 



Number of GEDs awarded 
5000 




Reporting year 

Source: GAO analysis of data from AOC Solutions, Inc. 



Each month of the post-residential phase, each Challenge Program 
graduate, or that individual's mentor, reports on the graduate’s post- 
residential activities. Following the 12-month post-residential phase, each 
Challenge Program site reports graduate placements in continuing 
education, the military, or the labor force. These placements are verified 
with schools, the military, and employers and are documented. Program 
representatives are not always able to contact all graduates for placement 
information and therefore placement data reflect only the students 
contacted, not all graduates. Education placements include returning to 
high school or going to a post-secondary or vocational-training institution, 
which students may be attending full- or part-time. Some Challenge 
Program graduates also enter the military, into either the active or the 
reserve forces. Post-residential employment placements can be full- or 
part-time, and they include those graduates who are self-employed. 
Graduates can have placements in more than one of these categories. For 
example, an individual might be attending school and working part-time. 
Challenge Program sites continue to update their placement records after 
the 12-month follow-up period when they come in contact with former 
students. During the longitudinal study, for example, the contractor has 
been able to update placement data based on information received from 
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state program officials and graduates. Figure 6 shows post-residential 
placement trends for the past 3 reporting years. The total numbers of 
graduates placed in these 3 years are 2,407 in 2003; 3,698 in 2004; and 4,086 
in 2005. 



Figure 6: Post-Residential Placement Data, 2003-2005 



Number of students 
3000 




Reporting year 




Employment 

Education 

Military 

Miscellaneous 13 



Source: GAO analysis of data from AOC Solutions, Inc. 



Notes: Program representatives are not always able to contact all graduates for placement 
information and therefore placement data reflect only the students contacted, not all graduates. 

“According to DOD, data for 2005 are preliminary, may change, and have not yet been officially 
published in an annual report. 

"Miscellaneous placements include caregivers, volunteers, homemakers, incarcerated, or individuals 
who are unable to hold an approved placement due to disability and/or hospitalization. Graduates 
must care for an individual or work as a volunteer for a minimum of 30 hours per week to be 
considered placed as a caregiver or volunteer. 
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Reserve Affairs and 
NGB Have Not 
Analyzed Data to 
Determine Actual 
Program Costs or the 
Need to Adjust the 
Federal and State 
Cost Share 



Although Reserve Affairs and NGB have expressed concern about the 
current program funding level and have suggested increasing both the cost 
basis used to determine funding needs and the federal cost share, we 
found that neither Reserve Affairs nor NGB has performed analyses to 
support the need for such changes. Good budget practices, included in the 
Office of Management and Budget’s Federal Financial Accounting 
Standards , 13 state that agencies should determine actual costs of their 
activities on a regular basis and that reliable cost information is crucial for 
effective management of government operations. Without better cost and 
financial information, DOD cannot justify future funding requests or a 
change in the cost-share ratio. 

Other than calculating how inflation has affected program costs, NGB has 
not analyzed data on actual program costs. Since 1993, NGB has used a 
cost of $14,000 per student as the basis for determining the amount of 
funds needed to cover program operating costs. In 2003, NGB calculated 
that if that amount were adjusted for inflation, it would be $18,000. The 
results of our survey of all Challenge Programs showed that in 2004, states 
actually spent between $9,300 and $31,031 per graduate with an average of 
$15,898 per graduate. 14 In addition, our survey showed that, on average, 
states estimated that the program should be funded at approximately 
$16,900 per target graduate to cover all of the services in the cooperative 
agreements, although the estimates ranged from $14,000 to $31, 800. 15 Most 
Challenge Program officials also told us that increasing the cost per 
student funding level for the program without increasing the federal cost 
share would negatively impact their programs because their states would 
be unlikely to come up with the additional state match money. Because 
costs vary between states due to regional differences in salary levels, staff 
benefits, and facility costs, Reserve Affairs has asked NGB to determine a 
new funding formula for the program based on individual state needs. At 



13 Office of Management and Budget, Statement of Federal Financial Accounting 
Standards, Number 4, Managerial Cost Accounting Concepts and Standards for the 
Federal Government, (Washington, D.C.: July 31, 1995). 

14 We calculated this number with information provided to us by individual states on our 
survey. According to our calculations, in 2004, the Fort Gordon, Georgia Program spent 
$9,300 per graduated student and Alaska spent $31,031 per graduated student. In order to 
cover the high personnel costs in Alaska and to provide program enhancements, the state 
contributes more than its required 40 percent share. 

15 Five programs said that the current funding level of $14,000 per graduate was sufficient: 
Ft. Gordon, Georgia; Camp Beauregard, Louisiana; Montana; New Mexico; and Puerto Rico. 
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the time of our review, NGB had not yet done this and Reserve Affairs has 
not given NGB a deadline for completion. 

In addition to expressing a desire to change the amount of funding per 
student, Reserve Affairs, NGB, and participating states have suggested that 
the cost-share ratio be changed from its current 60 percent federal share to 
a 75 percent federal share because they believe that the current 40 percent 
state share is sometimes difficult for states to meet; however, neither 
Reserve Affairs nor NGB has analyzed states’ financial situations or the 
impact of adjusting the federal and state cost share. Challenge Program 
officials told us that increasing the federal cost share of the program 
would be beneficial because it would enable states to expand their 
existing programs; give states more flexibility in funding their programs; 
and allow programs to restore to students some services that had been 
eliminated due to budgetary constraints. Although we did not analyze how 
changing the cost basis or the cost-share ratio would affect specific states, 
we prepared hypothetical examples for illustrative purposes. Table 1 
shows how changing the cost basis and the cost-share ratio would affect 
federal and state required funding levels. 



Table 1: Hypothetical Examples Showing How Changing the Cost Basis or Cost-Share Ratio Would Affect Yearly Challenge 
Program Funding Amounts 







Per individual program 




Total Challenge Program 




Typical 
number of 
students per 
program 


State cost 
share per 
program 


Federal cost 
share per 
program 


Total 
number of 
students in 
program 8 


Total 
required 
state cost 
share 


Total 
required 
federal cost 
share 


Total 

program 

cost” 


Current funding situation 
of $14,000 per student 
and 60 percent federally 
funded 


200 


$1,120,000 


$1,680,000 


7,000 


$39,200,000 


$58,800,000 


$98,000,000 


Change in cost basis to 
$16,900° per student and 
60 percent federally 
funded 


200 


$1,352,000 


$2,028,000 


7,000 


$47,320,000 


$70,980,000 


$118,300,000 


Change in funding ratio 
to 75 percent federally 
funded with cost basis of 
$14,000 per student 


200 


$700,000 


$2,100,000 


7,000 


$24,500,000 


$73,500,000 


$98,000,000 


Change in cost basis to 
$16,900° per student and 
75 percent federally 
funded 


200 


$845,000 


$2,535,000 


7,000 


$29,575,000 


$88,725,000 


$118,300,000 



Source: GAO analysis. 
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“Approximate number of students currently in program. 

"This amount does not include NGB overhead costs. 

“This amount is based on the average amount that states estimated in our survey would cover all of 
the services outlined in the cooperative agreements. 

In our survey, states reported varying views on whether they were 
experiencing difficulty in meeting their share of program costs. We did 
not verify the basis for their responses. Our survey showed that some 
states are able to provide funds above the required match; some states 
provide only the required match; and some states are unable to provide a 
match based on $14,000 per student and therefore fund the program at a 
lower level and receive less money from NGB. For example, California is 
able to provide additional support beyond the required match through 
additional money provided from the state general fund and funding from 
the program’s local school district. In 2004, California spent 
approximately $20,200 per graduate. Oregon, on the other hand, funds its 
program primarily through state education money. Due to recent state 
budget difficulties, Oregon cannot fund the program at $14,000 per 
student. In 2004, Oregon spent approximately $12,600 per graduate. 
According to the National Guard Bureau, all 29 programs are providing the 
services required by the cooperative agreements, and several states have 
added program enhancements such as field trips or vocational classes. 
However, some states reported that they reduced nonrequired services to 
stay within their budgets. For example, they implemented pay and hiring 
freezes; eliminated the student stipend; and eliminated program 
enrichment activities, such as field trips and vocational classes. Some 
states told us that additional funding, provided by a change in the cost- 
share ratio or an increase in the per student funding amount, would allow 
them to restore some of these services. 
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The National Guard 
Bureau Lacks a 
Complete Framework 
for Oversight of the 
Challenge Program, 
Making It Difficult to 
Measure Program 
Effectiveness 



Although NGB has several mechanisms in place for overseeing the 
Challenge Program, it lacks a complete oversight framework, making it 
difficult to measure the effectiveness of the program. A complete 
oversight framework, as suggested by the Government Performance and 
Results Act of 1993 (the Results Act) 16 and Standards for Internal Control 
in the Federal Government , 17 includes performance goals and measures 
against which to objectively measure performance as well as a mechanism 
for tracking findings of audits or reviews and responding to those findings. 
Currently, NGB does not require participating states to establish 
performance goals for individual programs and therefore does not have a 
firm basis for evaluating program outcomes and DOD’s return on 
investment. In accordance with the cooperative agreements, U.S. property 
and fiscal officers in each state are required to conduct full audits of state 
Challenge Programs at least every 3 years. However, these audits have not 
been conducted as required; and, when audits are conducted, copies of the 
results are not provided to NGB for review. Without regular audits and 
access to results, NGB cannot be assured that programs are using federal 
funds appropriately and that audit findings are addressed. 



The National Guard 
Bureau Conducts Several 
Oversight Activities and 
Recently Implemented a 
Mechanism for Tracking 
Audit Findings 



NGB conducts several oversight activities for the Challenge Program. In 
accordance with the DOD Instruction 18 and cooperative agreements 
governing the Challenge Program, NGB is responsible for the overall 
administration of the program, including program oversight. NGB uses 
both informal and formal mechanisms to oversee the program. On an 
informal basis, NGB frequently communicates with state program 
directors via e-mail and telephone calls. In addition, according to NGB, if 
a program director has a problem or an issue that he or she feels NGB 
needs to be involved with, he or she will initiate contact. Formal oversight 
of the program is conducted by NGB through AOC Solutions with yearly 
operational evaluations and biennial resource management reviews of all 
29 Challenge Programs. The purpose of the operational evaluations is to 
assess the programs’ compliance with the cooperative agreements and the 
implementation of the eight core components. The resource management 
reviews focus on assessing programs’ financial accountability and 



16 Pub. L. No. 103-62 (1993). 

^GAO, Standards for Internal Control in, the Federal Government, GAO/AIMD-OO-21.3.1 
(Washington, D.C.: November 1999). 

18 Department of Defense, Instruction 1025.8, National Guard Challenge Program , March 
20, 2002. 
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reviewing resources including staffing levels and salaries; food service 
costs; and physical inventory of property. 

Both the operational evaluations and the resource management reviews 
have identified areas for improvement and, according to NGB, changes 
were made to the program. For example, the operational evaluation of 
one state program conducted in fiscal year 2004 reported that program 
staff was calling the students inappropriate names. According to NGB, a 
staff member was dismissed as a result of this finding. Another 
operational evaluation of a different program conducted in fiscal year 2005 
found that over 90 percent of the community service hours accumulated 
by the students were for kitchen patrol. The cooperative agreements state 
that work in the dining facility may not be counted towards community 
service hours. According to NGB, this program has completely revamped 
its community service program and students now participate in such 
activities as visiting with residents of the local veterans’ home and caring 
for a historic cemetery. 

Until recently, NGB did not have a formal mechanism for tracking the 
findings of the reports conducted by AOC Solutions. During our review, 
we discussed with NGB the importance of keeping track of review findings 
in order to adequately respond to these findings, in accordance with 
Standards for Internal Control in the Federal Government. In response 
to our review, in October 2005, NGB provided new guidance to AOC 
Solutions regarding the operational evaluations and resource management 
reviews, which required, among other things, AOC Solutions to review 
findings from previous evaluation reports to determine whether corrective 
actions have been taken where warranted. In addition, officials at NGB 
told us that they currently monitor responses to audit and review findings 
informally with individual program directors. 



The National Guard 
Bureau Does Not Require 
Participating States to 
Establish Performance 
Goals for the Challenge 
Program 



Although NGB requires state Challenge Programs to report on certain 
outcome measures, such as GED attainment and number of graduates, 
NGB does not require states to establish any performance goals in these 
areas to measure the effectiveness of the program. The establishment of 
performance goals is consistent with the principles of effective 
management as set forth in the Results Act and would allow NGB to better 
evaluate the overall performance of the program and to assess DOD’s 
return on its investment. Without clear and agreed upon performance 
goals, there is no objective yardstick against which to fully measure 
program performance and thereby assess DOD’s return on investment. 
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Although it may not be reasonable to have the same performance goal for 
all state programs, it would be appropriate for each state program to 
negotiate a performance goal for defined performance areas such as 
increases in standardized test scores or physical fitness levels. Similar 
state programs, overseen by the Department of Labor, set individual 
negotiated levels of performance for specified core performance 
measures. These measures are used to provide information for 
systemwide reporting and evaluation for program improvement. For 
example, state Workforce Investment Act 19 programs negotiate 
performance measures for youth ages 14 to 18 in three areas: attainment of 
basic skills; attainment of high school diplomas or their equivalents; and 
placement in education, the workforce, or the military. For the area of 
diploma attainment, the goals range from 42.8 percent of participants in 
Louisiana to 68 percent of participants in New Hampshire. 

Currently, Challenge Program states are required to submit state plans 
annually. These state plans are required to contain long-term and annual 
performance goals and are to be updated annually. However, NGB has not 
provided guidance on specific performance areas where states should 
focus their goals; therefore, states may not have goals in the same 
performance areas, making it difficult for NGB to compare performance 
across programs. For example, California’s state plan contains a goal to 
acquire additional sources of funding through grants and charitable 
contributions. Oregon’s state plan, on the other hand, contains a goal 
stating that 80 percent of graduates from the residential portion of the 
program will be placed in education, the military, or employment but does 
not contain any goal related to acquiring additional sources of funding. In 
addition, states are not currently held accountable to the goals that they 
do set since the evaluation process does not measure the states’ 
performance against their goals. NGB told us that beginning in January 
2006, states would be held accountable to the goals outlined in their state 
plans. 



19 The Workforce Investment Act of 1998 was passed with the goal of consolidating, 
coordinating, and improving employment, training, literacy, and vocational rehabilitation 
programs in the United States, Pub. L. No. 105-220 (1998). Youth programs under the 
Workforce Investment Act seek to assist youth with employment and educational goals and 
the services needed to achieve those goals. 
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U.S. Property and Fiscal 
Officers Have Not 
Conducted Required 
Audits and Audit Results 
Are Not Provided to NGB 



Although the cooperative agreements governing the Challenge Program 
currently require U.S. property and fiscal officers to perform full audits of 
state Challenge Programs at least every 3 years, and prior to January 2005, 
the cooperative agreements required full audits every year, these audits 
have not been conducted as required. For example, according to NGB, out 
of a required 29 audits, only 14 were conducted in 2003, and only 7 audits 
were conducted in 2004. According to property and fiscal officers that we 
spoke with, audits were not conducted due to the lower priority placed on 
these audits compared to other audits that needed to be conducted within 
the state and a lack of staff to conduct the audits. Because the property 
and fiscal officers are responsible for ensuring that federal dollars are 
appropriately spent, if these audits are not conducted, it may be difficult to 
ensure that federal interests are adequately protected. 

When the property and fiscal officers do conduct audits of the Challenge 
Program, they are not currently providing copies of the audit results to 
NGB because, according to the Chief of Property and Fiscal Affairs at 
NGB, there is no specific requirement to do so. Standards for Internal 
Control in the Federal Government states that agencies need to ensure 
that the findings of audits and reviews are promptly resolved. If NGB does 
not review these audits, it cannot ensure that audit findings are resolved or 
identify trends across programs that may require action at a programwide 
level. According to officials at NGB, the audits remain internal to the state 
and the property and fiscal officer works directly with the state Challenge 
Program site to resolve any issues. In addition, if any audit findings 
require action from NGB, the state property and fiscal officer will contact 
NGB and ask for assistance. NGB needs to be aware of all audit findings, 
including those reported by the property and fiscal officers to effectively 
manage the Challenge Program. 

We reviewed the most recent property and fiscal officer audits for each 
Challenge Program participating state and found that they did identify 
areas that needed improvement. For example, an audit conducted of one 
program in fiscal year 2004 found unspent program dollars totaling 
approximately $180,000 that needed to be returned to the NGB. According 
to a follow-up report by the property and fiscal officer, this money was 
returned to NGB. Another audit conducted during fiscal year 2003 found 
that oversight of budget expenditures for one program was not adequate 
to ensure accurate, timely, and complete accounting of expenditures and 
recommended that key management controls over the program be 
identified, documented, published and tested. A follow-up audit of the 
program found that this recommendation was implemented. 
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Reserve Affairs Lacks 
a Formal Approach 
for Securing 
Alternative Funding 
Support for the 
Challenge Program, 
While Some 
Participating States 
Have Obtained 
Alternative Funding 
Support as a Result of 
Their Efforts 



Reserve Affairs has not adopted a formal strategy for pursuing nondefense 
funding, while some participating states have obtained alternative funding 
support for their programs as a result of their own efforts. Because 
Reserve Affairs has not made a formal business case to request funds from 
nondefense agencies, these agencies are unable to determine whether or 
not they are able to fund the Challenge Program. As a result, Reserve 
Affairs is potentially missing out on additional sources of funding that 
could enhance the program. Some participating states have obtained 
alternative funding support to enhance or maintain services provided to 
Challenge Program students. 



Nondefense Agencies Are 
Unable to Determine If 
They Can Fund the 
Challenge Program 
without a More Formal 
Request for Funding 



Reserve Affairs has not made a formal business case to request funds from 
nondefense agencies and therefore these agencies are unable to determine 
whether or not they are able to fund the Challenge Program. Although the 
authorizing legislation for the Challenge Program allows the Secretary of 
Defense to use nondefense funding sources in support of the program, 
Reserve Affairs has not adopted a formal strategy for pursuing nondefense 
funding. Rather, Reserve Affairs has primarily adopted informal strategies 
to contact agencies outside DOD to inform them about the Challenge 
Program and seek opportunities for partnerships. For example, Reserve 
Affairs officials have sent information via e-mail to officials at federal 
agencies with programs targeting at-risk youth (i.e. Department of Justice 
(DOJ), Department of Labor (DOL), and Department of Education (ED)) 
to inform them about the Challenge Program. These e-mails did not 
include specific requests for program funding. In addition, in July 2003, 
Reserve Affairs presented senior DOL officials with a proposal for forming 
an interagency partnership with the Challenge Program, but did not ask 
for a commitment of funding from DOL in a specified amount. 

Additionally, officials from these agencies have been invited to events 
sponsored by the National Guard Youth Foundation in support of the 
Challenge Program. Lastly, Reserve Affairs officials told us that they 
participate or have participated on a number of interagency councils and 
working groups that represent at-risk youth such as the President’s Crime 
Prevention Council, the White House Federal Interagency Working Group 
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on Service, the Corporation for National and Community Service, the 
National Civilian Community Corps, the Math Science Initiative, and the 
Juvenile Justice and Delinquency Prevention Council. 30 

Officials from DOJ, DOL, and ED noted that their agencies have general 
authority to provide funds to other programs if the transferred funds were 
used to support a program consistent with their agency’s interests. It is 
not unprecedented for an agency to transfer funds to the Challenge 
Program. In fiscal years 2000 and 2001, DOD and DOJ signed an 
interagency agreement under which DOJ agreed to provide $5 million in 
2000 and $7,483,500 in 2001 to the Challenge Program. 21 As DOJ concluded 
in this situation, and as officials at DOJ, DOL and ED stated, the transfer of 
funds to another agency must be consistent with the purpose and legal 
requirements of the program from which funds would be transferred. 
Moreover, these officials stated that sufficient funding must be available in 
the program from which the funds would be transferred, and making such 
a transfer must be in the interest of the agency. 

Officials at DOJ, DOL, and ED stated that Reserve Affairs needed to 
present more specific information to them before they could make a 
determination as to whether funds could be provided to the Challenge 
Program. These officials stated that an executive branch agency that 
wishes to make a request for funding from another executive branch 
agency could make such a request in any number of ways, either formally 
or informally. At a minimum, they noted, any such request should contain 
the amount of funding sought and a sufficiently detailed description of the 
program to allow the agency receiving the request to determine whether it 
would be an appropriate use of funds. At the time of our review, none of 
the agency officials we met with were aware of any specific request from 
Reserve Affairs concerning this matter. Until Reserve Affairs makes a 



30 As of 1997 and 2000, respectively, the President’s Crime Prevention Council and the White 
House Federal Interagency Working Group on Service no longer hold meetings. 

21 Specifically, the interagency agreement was signed between the Department of Defense 
and the Department of Justice, Office of Justice Programs, Office of Juvenile Justice and 
Delinquency Prevention. The authority for this transfer of funds was found in the Juvenile 
Justice and Delinquency Prevention Act of 1974, as amended (codified at 42 U.S.C. §5665). 
According to the interagency agreement signed in 2001, the funds were to be used to fund 
up to eight new Challenge Programs; reduce negative behaviors of program youth including 
delinquency, violence, substance use, school suspensions and expulsions, and unplanned 
teen pregnancies; and to engage up to 200 youth in each of the program sites in 
constructive programming. 
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more formal request for funding, other agencies will be unable to 
determine if they are able to provide funds for the Challenge Program. 



Some Participating States 
Have Obtained Alternative 
Funding Support to 
Enhance or Maintain 
Services Provided to 
Challenge Program 
Students 



Under cooperative agreements with NGB, states are not required to seek 
funding support beyond the required federal and state contributions; 
however, some states have made efforts to obtain alternative funding 
support for their programs to enhance or maintain the services provided to 
Challenge Program students. Under authorizing legislation for the 
Challenge Program, states may accept, use, and dispose of gifts or 
donations of money, other property, or services for the Challenge 
Program. Reserve Affairs and NGB officials said that they encouraged 
states to seek out additional funding sources for their programs if they 
want to enhance the services provided to program students. 



To assist state programs in their efforts to obtain that additional support, 
NGB, through its contractor Dare Mighty Things, shares information 
among state programs on strategies for successfully organizing a 501 (c) 
(3) corporation, 22 developing a fundraising policy that focuses on the long- 
term vision of the Challenge Program, accessing and sharing a grant writer 
with other programs, applying for National School Lunch Program funds, 
and educating state legislators to secure funding support. By providing 
information and examples of programs that are currently implementing 
these strategies, other state Challenge Programs have the information 
available to them on how to obtain additional funding support for their 
programs. In our review of the program, we found that some states had 
identified strategies for soliciting additional funding support from 
nonprofit organizations. For example, in the annual plans submitted to 
NGB, some states developed specific strategies for obtaining additional 
funding support such as setting a goal for the number of grants they would 
apply for and establishing a 501 (c) (3) corporation to raise funds on 
behalf of the program. 

We saw further evidence of states’ efforts to obtain additional funding 
support through our site visits and in our survey of the 29 Challenge 
Program sites. For example, in one state we visited, the Challenge site 
was also a charter school, which allowed the program to receive 



22 According to the Internal Revenue Code, a 501 (c) (3) corporation is an organization 
exempt from taxation, and organized and operated exclusively for religious, charitable, 
scientific, testing for public safety, literary, or educational purposes (26 U.S.C. §501 (c) 
(3))- 
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additional funding from the local school district. Moreover, in four states 
we visited, the state legislatures provided additional funding beyond the 
required state match of 40 percent to pay for additional staffing costs or 
facilities. According to our survey of all Challenge Programs, 28 out of 29 
programs identified receiving some type of funding support beyond the 
required federal and state contributions to pay for program expenditures 
incurred during 1999 through 2004. These programs identified receiving 
additional funding from the states’ general fund beyond the required state 
match as well as support from other state agencies such as the state 
Department of Education. Additionally, the programs we surveyed relied 
on funding assistance from other federal agencies such as the Department 
of Agriculture, which provides funding under the National School Lunch 
Program; nonprofit organization grants; the programs’ private 501 (c) (3) 
corporation; and donations from private individuals. 

Results from our site visits and national survey also showed that state 
Challenge Programs rely on donations of goods and services to support 
the program. For example, states we visited relied on donations from 
private citizens, corporations, schools, and the states’ National Guard units 
for such items as computers and software, exercise equipment, books, 
uniforms, and shoes. Some programs also received support from the 
surrounding community in the form of donated services such as 
transportation assistance and medical services from local doctors and 
nurses. Our survey further showed that 21 out of 29 programs received 
some form of donated goods and services. Program officials we met 
stated that donations of goods and services were vital to programs 
because many times they do not have funds available to acquire equipment 
or services for the program. Donations of goods and services can enhance 
the basic program beyond the military-based training and academics 
required under the cooperative agreements adding little or no cost to the 
program. 

Despite the success that many states had in obtaining additional funding 
support and donations of goods and services for their programs, some 
states reported obstacles in securing supplemental funding. For example, 
the cooperative agreements governing the Challenge Program do not allow 
states to use Challenge Program funds to hire full-time grant writers. 

Lastly, some programs we visited expressed concern that if they received 
funding support outside the federal and state required contributions, then 
their state governments might reduce their allocations. 
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Conclusions 



Although Reserve Affairs, NGB, and participating states have suggested 
the current cost basis of $14,000 per student is not sufficient to sustain the 
program and that the cost-share ratio should be changed, Reserve Affairs 
and NGB have done little analysis to show what the actual costs of the 
program are and how changing the cost-share ratio for the program would 
impact participating states. Without better cost and financial information, 
DOD cannot justify future funding requests or a change in the cost-share 
ratio. 

Until NGB establishes clear and agreed upon performance goals, there is 
no objective yardstick against which to fully measure program 
performance. Without these performance goals in place, NGB does not 
have a firm basis for evaluating program outcomes and DOD’s return on 
investment. 

Although property and fiscal officers are required to conduct full audits of 
the Challenge Program at least every 3 years, these audits have not been 
conducted due to competing priorities at the state level and a lack of staff. 
Without these audits, it may be difficult to ensure that federal interests are 
adequately protected. Also, because NGB does not receive copies of 
audits conducted by property and fiscal officers, it cannot know what the 
findings of these audits were and if changes were made to the programs 
based on these findings. 

Until Reserve Affairs makes a more formal request for funding, other 
agencies will be unable to determine if they can provide funds for the 
Challenge Program. Without a more formal process that outlines the 
amount of funding needed and a detailed description of the Challenge 
Program and its specific funding needs, Reserve Affairs is potentially 
missing out on additional sources of funding that could enhance the 
program. 



Recommendations 
Executive Action 



To improve the management and oversight of the National Guard Youth 
Challenge Program, we recommend that the Secretary of Defense, direct 
the Under Secretary of Defense for Personnel and Readiness, in 
consultation with the Assistant Secretary of Defense for Reserve Affairs 
and the Chief of the National Guard Bureau, to take the following three 
actions: 



• Determine the actual costs of the Challenge Program, including states’ 
ability to fund their share of the program, and use this information, as 
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appropriate, to support funding requests or a request to change the cost- 
share ratio. 

• Establish performance goals to measure the effectiveness of the Challenge 
Program. 

• Direct U.S. property and fiscal officers to conduct audits as required and 
require that copies of audit results are provided to the appropriate office 
at the National Guard Bureau in order to ensure that the results of audits 
are promptly reviewed and resolved. 

To strengthen efforts at obtaining alternative funding in support of the 
National Guard Youth Challenge Program, we recommend that the 
Secretary of Defense direct the Undersecretary of Defense for Personnel 
and Readiness, in consultation with the Assistant Secretary of Defense for 
Reserve Affairs, to develop more formal strategies for requesting 
alternative funding support for the Challenge Program. Such strategies 
may include submitting requests for funding that include the amount of 
funding requested and a sufficiently detailed description of the proposed 
program to allow potential providers of funds, such as nondefense 
agencies, to determine whether it would be an appropriate use of their 
funds. 



Agency Comments 
and Our Evaluation 



In written comments on a draft of this report, DOD concurred with our 
recommendations. In its overall comments, DOD asserts that our report 
reaffirms that the program met its congressional mandate to improve the 
life skills and employment potential of participants by providing military- 
based training and supervised work experience. Further, DOD’s 
comments state that our report validates the core program components of 
assisting participants to receive a high school diploma or its equivalent; 
developing leadership skills; promoting fellowship and community service; 
developing life coping and job skills; and improving physical fitness, health 
and hygiene. DOD also states that it appreciates GAO’s confirmation that 
it is properly executing the program. 

We do not agree with DOD’s characterization of our report. Whereas our 
report provides background data on program performance and mentions 
that DOD has reported positive outcomes from the Challenge Program, we 
do not assess whether or not DOD met its congressional mandate; we do 
not validate the program’s core components, nor do we confirm that DOD 
is properly executing the program, as DOD’s comments suggest. Rather 
our objectives were to examine: (1) historical trends of the National Guard 
Youth Challenge Program, including program expenditures, participation, 
and performance; (2) the extent to which Reserve Affairs and the NGB 
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have determined actual program costs and the need to adjust the federal 
and state cost-share; and (3) the extent to which the NGB has provided 
oversight of the program. We also determined the extent to which Reserve 
Affairs and participating states have made an effort to obtain alternative 
funding support for the Challenge Program. 

In response to our recommendation to determine the actual costs of the 
Challenge Program, including the ability of states to fund their share, DOD 
concurred, but claims that the matching fund requirement makes the 
program vulnerable to state budget cuts. DOD also contends that until 
recently, sluggish state revenues made the cost share requirement more 
burdensome and inhibited the ability of both the NGB and the states to 
focus on determining the actual costs of the Challenge Program. DOD 
claims that the budget shortfalls required the states to reduce and/or 
eliminate program services and that, with state revenues slowly closing 
their budget gaps, restoring funding for needed program services is being 
considered. DOD also notes that it is currently working on a new funding 
formula for the program. 

Although DOD’s comments suggest that adequate funding was not 
available for needed services, through our survey and other work, we 
found that all 29 programs were able to provide the services required by 
the cooperative agreements, and several states reported they added 
program enhancements such as field trips or vocational classes. However, 
some states reported that they reduced nonrequired services to stay within 
their budgets. Furthermore, the basis for DOD’s comment regarding the 
effect of the matching fund requirement and the level of state revenues on 
the program is unclear. As we reported, neither Reserve Affairs nor NGB 
has compiled or analyzed data on actual program costs, states’ financial 
situations, or the impact of adjusting the federal and state cost share. We 
continue to believe that any future funding request, or a request to change 
the cost share requirement, should be based on an analysis of the actual 
costs of the program. 

DOD’s comments are printed in their entirety in appendix III. 



We are sending copies of this report to the Secretary of Defense and 
interested congressional committees. We will also make copies available 
to others upon request. This report will be available at no charge on 
GAO’s Web site at http://www.gao.gov. 
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If you or your staff have any questions regarding this report, please 
contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on the 
last page of this report. GAO staff who made key contributions to this 
report are listed in appendix IV. 




Sharon L. Pickup 

Director, Defense Capabilities and Management 
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Appendix I: Scope and Methodology 



To examine historical trends of the National Guard Youth Challenge 
Program, including program expenditures, participation, and performance, 
we interviewed officials from the Office of the Assistant Secretary of 
Defense for Reserve Affairs; the National Guard Bureau (NGB); and the 
contractor that monitors and evaluates state programs, AOC Solutions. 

We reviewed other documentation provided by Reserve Affairs and NGB, 
such as program funding summaries. We also reviewed the Challenge 
Program’s annual reports submitted to Congress from 1994 through the 
present, excluding 1997, 1 which provided us background information on 
the program. For data on participation and performance outcomes, 
including general education development credential attainment, 
community service hours, and post-residential placements, we relied on 
the Data Management and Reporting System (DMARS) that is used by 
NGB to collect participant information and track individual and program 
activities. DMARS was implemented for the 2003 reporting year, and the 
contractor has performed procedures to clean up the data as far back as 
2000. We conducted data reliability tests on the program’s annual reports 
and data management system to conclude that the data are sufficiently 
reliable for our purposes. We did not compare the costs or outcomes of 
this program to other similar youth programs currently funded by the 
federal government since that was not within the scope of this 
engagement. 

To assess the extent to which Reserve Affairs and NGB determined actual 
program costs and analyzed states’ ability to fund their share of the 
program, we interviewed officials at Reserve Affairs and NGB as well as 
officials at state programs. We reviewed and analyzed internal budget 
documents prepared by state programs and submitted to NGB annually as 
well as documents prepared by individual program directors that 
discussed the funding situation in their state. We also reviewed and 
analyzed good budget practices as described in federal financial 
accounting standards. Using a semi-structured questionnaire, we 
interviewed eight state Challenge Program directors and their budget 
officers in seven states and discussed sources of funding for their 
programs, actual costs of operating their programs, and the types of 
services that are provided to program participants given the available 
funding. States selected for site visits or telephone interviews are shown 
in table 2. There were a number of factors which affected the judgmental 



According to NGB, there was no annual report published for 1997 because program 
officials were not sure whether or not Congress would permanently authorize the program. 
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Appendix I: Scope and Methodology 



sampling of these state programs. States were chosen because of their 
length of time in the program, size, geographic location, experience of the 
program’s director, and whether or not there were multiple programs in 
the state. Additionally, in July 2005, we surveyed all 29 participating 
Challenge Programs and asked them about the actual costs of their 
programs and their assessment of their states’ ability to fund the programs. 
This survey had a response rate of 100 percent. 



Table 2: States Selected for Site Visits or Interviews 


State 


Program Name 


Arizona 


Arizona Project Challenge 


California 


California Grizzly Youth Academy 


Illinois 


Illinois Lincoln's Challenge Academy 


Maryland 


Maryland Freestate Challenge Academy 


Oregon 


Oregon National Guard Youth Challenge Program 


South Carolina 


Camp Long 


South Carolina 


Columbia 


Wisconsin 


Wisconsin National Guard Challenge Academy 



Source: GAO. 



To assess the extent to which NGB has provided oversight of the 
Challenge Program, we interviewed officials at Reserve Affairs and NGB 
as well as program officials at the state level. To determine the oversight 
responsibilities of federal and state officials involved in the program, we 
reviewed and analyzed pertinent laws, the Department of Defense’s (DOD) 
Instruction governing the program, and the NGB Master Cooperative 
Agreement, which defines the terms and conditions of the program in each 
state. We also interviewed the independent contractor hired by NGB to 
conduct yearly and biennial on-site program reviews, AOC Solutions. We 
obtained access to its data management system and analyzed the program 
evaluations and resource management reviews it completed on behalf of 
NGB since 2003. We also reviewed and analyzed individual state 
program’s annual plans and NGB’s strategic plan to identify the types of 
goals set by the Challenge program. We consulted previous GAO work 
regarding performance measurement and evaluations, identified best 
practices for establishing and measuring performance goals, and reviewed 
Standards for Internal Control in the Federal Government and the 
Government Performance and Results Act of 1993. We interviewed 
representatives of the U.S. Property and Fiscal Officer’s Office in six states 
as well as the NGB Chief of Property and Fiscal Affairs to determine the 
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Appendix I: Scope and Methodology 



time frame for completing audits and mechanisms for reporting, tracking, 
and resolving issues that arise out of the PFO audits. 

To determine the extent to which Reserve Affairs and participating states 
have obtained alternative funding support for the program, we reviewed 
relevant laws, policies, and reports to determine the relevant authorities 
for receiving and transferring funds between federal agencies. We also 
interviewed federal officials at Reserve Affairs and the Departments of 
Labor, Education, and Justice to determine the extent to which these 
agencies discussed the possibility of transferring funds to DOD in support 
of the Challenge program. To determine states’ efforts at obtaining 
alternative funding support for their program’s operations, we conducted a 
survey of all 29 participating programs to collect information about their 
sources of funding for the program, how funding is distributed across 
different program operational functions, the sources and types of donated 
goods and services, and descriptions of strategies programs use to obtain 
alternative funding support. Using a semi-structured questionnaire, we 
interviewed eight state Challenge Program directors and members of their 
staff in seven states and discussed sources of funding for their programs, 
the strategies these specific programs use for obtaining alternative funding 
support and donations, and the difficulties these states face in seeking out 
other funding. 

We provided a draft of this report to officials at DOD for their review and 
incorporated their comments where appropriate. We conducted our work 
from January 2005 to October 2005 in accordance with generally accepted 
government auditing standards. 
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Appendix II: National Guard Youth Challenge 
Program Sites 



State 


Program name 


Calendar year first 
class began 


Number of 
graduates in 2004 


Alaska 


Alaska Military Youth Academy 


1994 


192 


Arizona 


Arizona Project Challenge 


1993 


182 


Arkansas 


Arkansas National Guard Youth Challenge Academy 


1993 


181 


California 


California Grizzly Youth Academy 


1998 


204 


Florida 


Florida Youth Challenge Academy 


2001 


227 


Georgia 


Georgia - Fort Gordon Youth Challenge Academy 


2000 


276 




Georgia - Fort Stewart Youth Challenge Academy 


1993 


444 


Hawaii 


Hawaii National Guard Youth Challenge Academy 


1994 


207 


Illinois 


Illinois Lincoln's Challenge Academy 


1993 


785 


Kentucky 


Kentucky Bluegrass Challenge Academy 


1999 


160 


Louisiana 


Louisiana Youth Challenge Program - Camp Beauregard 


1993 


407 




Louisiana Youth Challenge Program - Camp Minden 


2002 


214 




Louisiana Youth Challenge Program - Gillis W. Long 


1999 


331 


Maryland 


Maryland Freestate Challenge Academy 


1993 


224 


Michigan 


Michigan Youth Challenge Academy 


1999 


174 


Mississippi 


Mississippi National Guard Youth Challenge Program 


1994 


404 


Montana 


Montana Youth Challenge Program 


1999 


144 


New Jersey 


New Jersey Youth Challenge Program 


1994 


178 


New Mexico 


New Mexico Youth Challenge Academy 


2001 


197 


North Carolina 


North Carolina Tarheel Challenge Academy 


1994 


200 


Oklahoma 


Oklahoma Thunderbird Youth Academy 


1993 


221 


Oregon 


Oregon National Guard Youth Challenge Program 


1999 


221 


Puerto Rico 


Puerto Rico Youth Challenge Program 


1999 


208 


South Carolina 


South Carolina Youth Challenge Academy - Camp Long 


2002 


108 




South Carolina Youth Challenge Academy - Columbia 


1998 


257 


Texas 


Texas Seaborne Challenge Corps 


1999 


188 


Virginia 


Virginia Commonwealth Challenge Program 


1994 


140 


West Virginia 


West Virginia Mountaineer Challenge Academy 


1993 


175 


Wisconsin 


Wisconsin National Guard Challenge Academy 


1998 


154 




Total Graduates 




7,003 


Source: GAO analysis of data from AOC Solutions, Inc. for NGB Classes 21 and 22. 



Note: The Wyoming Youth Challenge Program received start-up funds in 2005, and will begin its first 
class in January 2006. Nevada does not have a Challenge Program site but sends approximately 24 
youths per year to the Arizona site. Until fiscal year 2005, the District of Columbia sent 60 youths per 
year to the Maryland site. 
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Appendix III: Comments from the 
Department of Defense 



ASSISTANT SECRETARY OF DEFENSE 

WASHINGTON, DC 20301-1500 

NOV 1 5 2005 

RESERVE AFFAIRS 

Ms. Sharon L. Pickup 

Director, Defense Capabilities and Management 
U.S. Government Accountability Office 
441 G Street, N.W. 

Washington, D.C. 20548 

Dear Ms. Pickup: 

This is the Department of Defense (DoD) response to the Government Accountability 
Office (GAO) draft report, “DEFENSE MANAGEMENT: Actions are Needed to Improve the 
Management and Oversight of the National Guard Youth Challenge Program,” dated October 24, 
2005 (GAO Code 350643/GAO-06-140). 

The report reaffirms that the Department met its congressional mandate to improve life 
skills and employment potential of participants by providing military-based training and 
supervised work experience. It also validated the core program components of assisting 
participants to receive a high school diploma or its equivalent, leadership development, 
promoting fellowship and community service, developing life coping skills and job skills, and 
improving physical fitness and health and hygiene. A more appropriate title might be “The 
National Guard Youth Challenge Program: Statutory Requirements Being Met - Management 
and Oversight Can Be Strengthened.” 

The Department appreciates GAO’s confirmation that the DoD is properly executing the 
National Guard Youth Challenge Program, and the suggestions for improvement. 

The DoD comments on the draft report findings and recommendations are enclosed. The 
Department of Defense appreciates the opportunity to comment on the draft report. If there arc 
any questions, my point of contact is Mr. Ernie Gonzales, (703) 693-8630. 

Sincerely, 

T.F. 

T. F. Hall 



Enclosure: 
As stated 
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Appendix III: Comments from the Department 
of Defense 



GOVERNMENT ACCOUNTABILITY OFFICE DRAFT REPORT DATED OCTOBER 24, 

2005 

(GAO CODE 35064 3/GAO-06- 1 40) 

‘DEFENSE MANAGEMENT: Actions Are Needed to Improve the Management and Oversight 
of the National Guard Youth Challenge Program” 



DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS 



RECOMMENDATION 1 : The GAO recommended that the Secretary of Defense, direct the 
Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant 
Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to 
determine the actual costs of the Challenge Program, including states’ ability to fund their share 
of the program, and use this information, as appropriate, to support funding requests or a request 
to change the cost-share ratio, (p. 34/GAO Draft Report) 

POD RESPONSE : Concur, but the matching fund requirement makes the Challenge Program 
vulnerable to state budget cuts. Until recently, sluggish state revenues made the cost-share 
burden more burdensome and prohibited both the National Guard Bureau and the states to focus 
on determining the actual costs of the Challenge Program. The budget shortfalls required the 
states to reduce and/or eliminate program services. With state revenues slowly closing their 
budget gaps, restoring funding for needed program services are being considered. In the report, 
the GAO affirmed that the Department is currently working on a new funding formula for the 
program based on individual state needs. This new formula will adequately address and 
determine funding for required program services. 

RECOMMENDATION 2 : The GAO recommended that the Secretary of the Defense, direct 
the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant 
Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to establish 
performance goals in order to measure the effectiveness of the Challenge Program, (p. 34/GAO 
Draft Report) 

POD RESPONSE : Concur, but although the Department does not specifically define program 
objectives, the GAO noted that the Department does report annually to Congress positive 
performance outcomes in academic performance, community service activities, and post- 
residential placements. 

RECOMMENDATION 3 : The GAO recommended that the Secretary of the Defense, direct 
the Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant 
Secretary of Defense for Reserve Affairs and the Chief of the National Guard Bureau to direct 
U.S. Property and Fiscal Officers to conduct audits as required and require that copies of audit 
results are provided to the appropriate office at the National Guard Bureau in order to ensure that 
the results of audits are promptly reviewed and resolved, (pp. 34 and 35/GAO Draft Report) 
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Appendix III: Comments from the Department 
of Defense 



POD RESPONSE : Concur. 

RECOMMENDATION 4 : The GAO recommended that the Secretary of the Defense direct the 
Under Secretary of Defense for Personnel and Readiness, in consultation with the Assistant 
Secretary of Defense for Reserve Affairs to develop more formal strategies for requesting 
alternative funding support for the Challenge Program. Such strategies may include submitting 
requests for funding that include the amount of funding requested and a sufficiently detailed 
description of the proposed program in order to allow potential providers of funds, such as non- 
defense agencies, to determine whether it would be an appropriate use of their funds. 

(p. 35/GAO Draft Report) 

POD RESPONSE : Concur. The Department acknowledges that it will need to take more 
formal actions for requesting supplemental funding and recognized that the informal meetings, 
presentations and discussions with other federal agencies did not result in formal commitments. 
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GAO’s Mission 


The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO’s 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 
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